Towards End-to-End Image Compression and Analysis with Transformers

We propose an end-to-end image compression and analysis model with Transformers, targeting to the cloud-based classification application. Instead of placing existing Transformer-based directly after codec, we aim redesign Vision Transformer (ViT) perform from compressed features facilitate long-term information Transformer. Specifically, first replace patchify stem (i.e., splitting embedding) ViT a lightweight encoder modelled by convolutional neural network. The generated are injected inductive bias fed for bypassing reconstruction. Meanwhile, feature aggregation module fuse selected intermediate Transformer, feed aggregated deconvolutional network can obtain self-attention mechanism improve performance. rate-distortion-accuracy optimization problem is finally solved two-step training strategy. Experimental results demonstrate effectiveness proposed in both tasks.